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ABSTRACT 



This study examined differences between two populations of 
composition students over the course of a semester in their perceptions of 
collaboration as reported in the pre- and post-surveys. In addition, using an 
assessment rubric developed by the researchers, the study examined students' 
audience awareness as demonstrated in their writing. Both groups of students 
were taught by the same instructor and had the same writing assignments; both 
classes incorporated a pedagogy of collaborative learning to help students 
develop a sense of belonging to a discourse community; and both sections used 
networked computers as learning, writing, and communication tools. The 
distinguishing variable between the two sections was the absence of 
face-to-face communication among students in the online class. The following 
research questions were posed in the study: (1) Do students differ in how 

they value collaboration during the writing process? (2) Do students differ 
in how they address audience needs in their writing products? (3) Do students 
differ in their level of satisfaction with their learning experience? 
Conclusions from this study are: online students tend to be more independent 
learners, valuing collaboration less than do on- campus students; online 
interaction appears to increase audience awareness in students' writing; and 
students in both sections reported positive experiences with their respective 
classes. (Contains 20 references.) (Author/AEF) 
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Abstract 

In this study, we examined differences between two populations of composition students over the course of 
a semester in their perceptions of collaboration as reported in pre^ and post^surveys. In addition, using an 
assessment rubric we developed, we examined students' audience awareness as demonstrated in their writing. Both 
groups of students were taught by the same instructor and had the same writing assignments; both classes 
incorporated a pedagogy of collaborative learning to help students develop a sense of belonging to a discourse 
community: and both sections used networked computers as learning, writing, and communication tools. The 
distinguishing variable between the two sections was the absence of face-to-face communication among students in 
the online class. In our study, we posed the following research questions: In a comparison of students in a on- 
campus networked writing class with students in an off-campus online writing class, (a) Do students differ in how 
they value collaboration during the writing process? (b) Do students differ in how they address audience needs in 
their writing products? (c) Do students differ in their level of satisfaction with their learning experience? 
Conclusions from this study are the following: (I) Online students tend to be more independent learners, valuing 
collaboration less than do on-campus students; (2) Online interaction appears to increase audience awareness in 
students' writing. (3) Students in both sections reported positive experiences with their respective classes. 

The field of composition has undergone two significant paradigm shifts in the last twenty years: from a 
focus on the product of writing to a focus on the process, and from a focus on the writer as a solitary individual to a 
focus on the writer as part of a discourse community, “a group of people with similar goals and interests who 
constitute themselves with a characteristic language” (Bruffee, 1993, p. 223). A more recent development, the use of 
computer-mediated instruction in the college composition classroom, has facilitated both paradigm shifts. 

At the University of Colorado at Colorado Springs (UCCS), for example, all writing classes are taught in 
networked-computer classrooms, where students interact both through face-to-face and through computer-mediated 
communication. In this environment, students have increased opportunities to engage in the writing process through 
collaborative paitnerships with other students. The networked computers become conduits for linking students with 
one another, thereby extending the discourse community in the classroom through a second framework via 
cyberspace: shared virtual “chambers” where continued interaction takes place and where a repository of student 
work, both individually and collaboratively produced, resides as a catalyst for further dialectic. 

At the same time, universities are experimenting with the delivery of writing courses exclusively via 
distance learning, without face-to-face interaction. In the fall 1999 semester, UCCS joined the ranks of these 
universities by delivering one section of freshman composition as an online course. 

Computer-Mediated Communication 

Computer-mediated communication (CMC) changes both the quality and quantity of communication by 
allowing time for critical reflection and greater involvement in discussion than is allowed in the traditional 
classroom, where one or two students may monopolize the conversation (Berge, 1995; Fishman, 1997; Wells, 1992). 
Group conferencing appears to decrease the emergence of a group leader, allowing more students a greater role 
(Harasim, 1990; Warschauer, 1997). Student-directed conversation and participation level is higher in the CMC 
classroom, which shifts the role of the teacher from content expert to facilitative guide (Wells, 1992). CMC thus 
enhances peer-to-peer discussion (Jonassen et al., 1995), and participation is fairly evenly distributed among 
students. Students report that they work harder and produce higher quality work, since work is visible to their peers 
(Oblinger & Maruyama, 1996). In addition, computer conferencing tools foster critical thinking and active learning 
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by providing an electronic space for reflective journal writing, critical analysis, and peer and instructor facilitation 
(Bonk & Reynolds, 1997). Instructors are able to observe students’ contributions to discussion, obtain a record of 
the discussion for future feedback, participate in the discussion to model critical-thinking skills, and ask questions to 
coach critical thinking, providing expertise when necessary (Wagner & McCombs, 1995). 

Collaboration 

Collaboration enhances connectivity and socio-emotional commitment to the learning process by involving 
students as active participants in the learning process (Sharan, 1980; Oliver & Reeves, 1994). Students achieve 
greater cognitive development working together than they do working individually (Sharan; Oliver & Reeves). 
Collaboration can contribute to the active construction of meaning, through idea generating (divergent thinking), 
idea linking (convergent thinking), and idea structuring (categorization and classification) (Harasim, 1990). Online 
collaboration, with its emphasis on both reflection and interaction, can amplify the process of knowledge 
construction (Warschauer, 1997). Collaborative problem solving, involving both conversation and issue-based 
discussion, supports intentional learning and develops critical-thinking skills (Duffy, Dueber, & Hawley, 1998). 
Web-based tools such as e-mail, electronic partnerships, project-based learning, and synchronous or asynchronous 
conferencing foster collaborative learning (Bonk & Reynolds, 1997). 

Advantages and Limitations of Asynchronous Collaboration 

While the effectiveness of these collaborative tools has not been extensively studied, asynchronous 
communication and online collaboration have some recognizable strengths and weaknesses. Strengths of computer- 
mediated collaboration include student enthusiasm, more time on task, and student satisfaction (Shotsberger, 1996; 
Kemer, Penner-Hahn, Berger, & Dershimer, 1997). Students appear to like CMC, find the instructor is more 
accessible, and find problem-based learning and case-study learning more useful than they are in a traditional 
classroom. Disadvantages include communication anxiety, feelings of disconnectedness from conversational thread, 
and frustration over delayed feedback. Additionally, making decisions from group consensus can be time-consuming 
(Harasim, 1990; Warschauer, 1997), while software and hardware problems may limit interaction (Oliver & Reeves, 
1994). One of the biggest drawbacks to asynchronous collaboration is the lack of visual and verbal cues provided by 
face-to-face interaction. (Lehman, 1995). 

Online Students vs. Traditional Students 

A comparison of online and traditional students shows that online students learn on average as well as 
traditional students, with respect to midterms, finals and grades. More mature and better students learn more, while 
students who lack good study habits and have difficulty writing and reading learn less (Harasim, 1990). Self- 
discipline is a crucial element of success in online learning. Hiltz (1990) examined learning in online and traditional 
classes using pre-/post-questionnaires, case studies, institutional data, interviews with students and faculty, and 
survey of dropouts. She found no significant difference between mastery in the online class and traditional 
classrooms. In fact, the grades for students in the online computer science class were better than the grades for 
traditional students. She concludes that online students learn as well as traditional students. Simonson, Schlosser, 
and Anderson (1994) concur: Students who are motivated, prepared, and intelligent can potentially learn as much 
online as in a traditional classroom. While attrition rates are generally higher for online students, researchers have 
found little correlation between performance outcomes and individual characteristics, especially for mature learners 
(Kember, 1990). 

Collaborative Learning, Writing, and Audience Awareness 

Collaborative learning in higher education has been underused and frequently misunderstood. Bruffee 
(1993) posits that knowledge is a social construct, that learning is a social process, and that writing is central — not 
ancillary— to collaborative learning and the construction of knowledge. In addition, Oakeshott (1962), Sergiovanni, 
(1996), Latour and Woolgar (1986) all emphasize the importance of social dialectic in the construction of 
knowledge. The use of technology in the teaching of writing has also been widely discussed (Hawisher, LeBlanc, 
Moran, & Selfe, 1996). However, although researchers have documented a variety of benefits from the integration 
of technology in the composition class (Carbone, 1993; Klem & Moran, 1992; Mason, Duin, & Lammers, 1994), the 
data about writing improvement are less clear. 

Current approaches to audience include historical studies (Willard & Brown, 1990; Willey, 1990), studies 
of writers’ audience awareness during the writing process (Moffett, 1968), studies of audience as a discourse 
community (Enos, 1990; Rafoth, 1990; Roth, 1990; Mangelsdorf, Roen, & Taylor, 1990), and links between 
audience awareness and syntactic and lexical features (Rubin & O'Looney, 1990). This interest in audience is related 
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to the increased focus on examining composition from a social constructivist perspective (Bruffee, 1986). In 
addition, an ongoing debate centers on whether writers “invoke” a fictionalized audience or “address” an actual 
audience. Scholars agree, however, that actual readers can have a powerful effect on writers (Long, 1990; Ede & 
Lunsford, 1 984; Porter, 1 992). 

Research Design and Methodology 

In this study, we examined differences between two populations of composition students over the course of 
a semester in their perceptions of collaboration as reported in pre- and post-surveys. In addition, using an assessment 
rubric we developed, we examined students' audience awareness as demonstrated in their writing. Both groups of 
students were taught by the same instructor and had the same writing assignments; both classes incorporated a 
pedagogy of collaborative learning to help students develop a sense of belonging to a discourse community; and 
both sections used networked computers as learning, writing, and communication tools. The distinguishing variable 
between the two sections was the absence of face-to-face communication among students in the online class. In our 
study, we posed the following research questions: In a comparison of students in a on-campus networked writing 
class with students in an off-campus online writing class, (a) Do students differ in how they value collaboration 
during the writing process? (b) Do students differ in how they address audience needs in their writing products? (c) 
Do students differ in their level of satisfaction with their learning experience? 

Our rationale for focusing on audience awareness was quite simple: One of the hallmarks of critical 
thinking and thus of good academic writing is the ability to examine an issue from various perspectives, to take into 
account opposing views, to be aware that one is writing not for oneself but for an audience of readers who have 
multiple perspectives and often considerable skepticism toward the writer’s perspective. The best way to increase 
one’s credibility with a skeptical audience is to acknowledge readers’ likely questions, concerns, and objections and 
to address them, summarizing opposing views fairly, and responding to those views either through concession or 
carefully crafted rebuttal (Rogers, 1961). Fundamentally, the writer’s obligation is not to create barriers between 
herself and the reader but rather to build bridges — to find common ground. As beginning writers, first-year college 
students have difficulty doing this, largely because they are locked in their personal perspectives, often viewing the 
world in a dualistic, “right and wrong,” lens (Perry, 1970; Perry, 1985). Collaboration during the writing process 
helps students expand their awareness of audience and thus, presumably, helps them improve this important aspect 
of their writing. 

Objectives of the freshman composition are to improve students’ research and argumentative writing skills 
and to help students gain confidence in their writing ability. Peer response sessions on papers-in-progress were an 
important course component, promoting collaborative learning and heightened audience awareness. To promote 
collaborative learning in both the on-campus and the online sections, students used FirstClass software, a 
communication and conferencing package that facilitates students’ ability to collaborate with peers and to engage in 
the various stages of the writing process. For the on-campus students, the computer classroom was equipped with 24 
networked PC workstations arranged around the perimeter of the classroom, along with an instructor workstation 
connected to a video display projector. Students in the on-campus section (N = 18) met twice a week in the 
networked classroom. Students in the online section (N = 15), on the other hand, with the exception of an initial and 
final on-campus meeting, completed all their interactions with peers and the instructor online. 

But whether students were enrolled in the on-campus or the online class, they were able to access their 
virtual classroom space, including assignments and work-in-progress by students in the class, both from home 
computers and from computers in open labs on campus. In both classes, the instructor gave the students a brief 
introduction and a written instructional guide to the technology itself Additionally, since this was a second-semester 
course, most of the students had had previous exposure to FirstClass in their first semester ofcomposition and were 
thus familiar with the software from the start of the semester. 

To determine demographic differences between the students in the on-campus and the online class, we 
compiled profiles of students in the two sections from survey questions (age, gender, GPA, grade level, work load, 
family status, technical expertise, and experience) and from the University Student Information System (SIS). 
Significance in this study was set at g = .05. 

Procedure for Measuring Students' Attitudes toward Collaboration 

Besides providing demographic information, students in both sections completed pre- and post-surveys 
(Fowler, 1993; Ehrmann & Zuniga, 1997) regarding their attitudes toward collaboration, responding to questions on 
a 6-point Likert-type scale. To increase face validity of these items, we distributed the instrument to twelve 
experienced composition faculty in the UCCS English Department Writing Program who reviewed the items and 
made suggestions. 
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Procedure for Measuring Students’ Performance in Audience Awareness 

To determine differences in demonstrated level of audience awareness, the first and final papers — out-of- 
class, research-based argumentative papers — were collected from all students in both sections. For control purposes, 
students were given the same assignments for the first and final papers. We wanted students to write on two topics at 
a similar level of “strength of opinion.” To determine this, we polled the students in the pre-survey, asking them to 
rank six topics on a scale of one to five, “one” representing indifference to a topic, and “five” representing a strongly 
held position on a topic. The two topics with the most similar means were gun control and capital punishment. Not 
only were the means similar (gun control topic X = 3.6429, 1-5 scale; capital punishment X = 3.6667, 1-5 scale), but 
students also had relatively strong positions on these issues, so we reasoned that these two topics would require 
similar levels of cognitive challenge for students in addressing an opposing audience. The first paper assignment 
instructed students to take and support a position on the issue of gun control, while the final paper assignment 
instructed them to take and support a position on the issue of the death penalty. In both cases, students were 
instructed to address an audience that disagreed with the position they took on the issue. The writer’s purpose was to 
gain the readers’ respect, if not their assent, for the position argued. 

The papers were coded using a random numerical coding system and were assessed by three experienced 
readers who first completed a "norming" of six of the papers, randomly selected (Elbow, 1996). The readers 
assessed the papers based on a nine-item rubric, using a primary-trait six-point criterion-referenced scale (Walvoord 
& Anderson, 1998) that we developed. The nine items include six elements of audience awareness important in 
argumentative writing, the genre focus of English 141: purpose, empirical support, logical appeal, ethical appeal, 
emotional appeal, and treatment of opposing views. These are based on Aristotle’s logos, ethos, and pathos, on 
Toulmin's (1958) model of informal reasoning, and on Rogerian rhetoric (Rogers, 1961). We also examined three 
additional elements considered standard in essay assessment: organization, syntax, and grammar. Trimble (2000) 
argues that these additional elements do in fact fall under the rubric of audience awareness, and that writing for an 
audience is less effective in the absence of control in these areas. Inter-rater reliability scores for the rubric elements 
ranged from 0.66 to 0.89. 

Results and Discussion 

The online class was significantly older (M =28.36 years) than the on-campus class (M = 20.5, F = 13.167, 
p < .01), and a significantly higher portion of the online students were married (p < .01). Although age was 
correlated with other demographic variables, such as the number of dependents, employment hours, and credit 
hours, these other variables were not significantly different between the two sections. 

Online students and on-campus students did not differ significantly in academic background. Students had 
comparable grade point averages, TSWE scores, SAT-English scores, ACT-English scores, and grades in English 
131, the prerequisite composition course for English 141. Students in both sections were similarly comfortable with 
computer technology, Internet access, and CyberClass usage. However, several interesting motivational differences 
between the sample populations were evident. Online students ranked themselves higher on self-discipline than did 
on-campus students, but they planned to devote fewer hours studying for the class. This difference is significant, 
even when age was used as a covariate (F = 6.473, p < .01). In addition, online and on-campus students had different 
motivations for taking the class that they selected. Online students cited convenience, while on-campus students 
cited the good time block as being the primary reason for choosing the particular section. 

Collaboration Survey Results 

At the beginning of the semester, students in both sections held similar views of collaboration, with no 
significant difference in overall collaboration scores between the two sections. However, online students' 
collaboration scores were significantly lower at the end of the semester than they had been at the beginning, 
suggesting a decrease in their valuing of collaboration as the semester progressed. This finding is in contrast to the 
literature that suggests that electronic communication enhances a sense of community (Harasim, 1990). In our study, 
online students valued community less, believed less strongly that knowing other students in the class improved 
learning, and exhibited less preference for face-to-face communication over written communication. 

Factor analysis of the 25 collaboration survey items resulted in two factors, accounting for 42.589% of the 
variance. The reliability for this instrument was a = .8910. Factor one (“valuing peer feedback on work in progress”) 
contained twelve items, while factor two (“sense of belonging to a discourse community”) comprised four survey 
items. Using AN OVA, we found no significant differences on either the pre- or the post-survey for factor one: 
Whether they were enrolled in the online or the on-campus section, no significant differences emerged in how 
students valued peer feedback on work in progress either on the pre-survey or the post-survey. We also calculated 
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the difference between sections in the amount of change in attitudes in factor one. While the scores for the online 
students went down over the course of the semester (M = -2.33) and the scores for the on-campus students went up 
(M = .357), the difference between groups in the amount of change was not significant (F =*1 .21, p < .282). 

Analysis of responses towards factor two items indicates that feeling connected was significantly less 
important to the learning experience for online students than it was for on-campus students. While the scores for the 
online students went down over the course of the semester (M = -1.38) and the scores for the on-campus students 
went up (M = 2.00), the difference in the amount of change was not significant (F = 2.45 1 , p < . 1 30). 

Audience Awareness Rubric Results 

In examining the survey results, we were primarily interested in students’ attitudes and self-perceptions. In 
examining students’ final papers, we shifted our gaze from attitudes to actual writing performance, focusing 
particularly on students’ demonstrated audience awareness. We wanted to see if any differences emerged in 
students’ writing depending on whether students collaborated on their papers in a face-to-face environment or 
exclusively online. 

We compared scores on the nine elements of the audience awareness rubric, on both the first and the final 
papers. In spite of the lack of face-to-face collaboration, online students scored significantly higher on eight of the 
rubric elements on the first paper and on all nine of the elements on the final paper, as can be seen in Table 1 : 

Table 1 

Comparison of Means in Audience Awareness Between Sections 



First Paper M Final Paper M 



Rubric 

Element 


Online 


ampus 


( MDif 


Online 


Campus 


MDif 


1 . Purpose 


13.07 


10.56 


2.51* 


13.60 


10.67 


2.93** 


2. Empirical 


13.20 


10.44 


2.76* 


14.33 


10.89 


3.44** 


evidence 


3. Logical appeal 


12.73 


10.33 


2.40* 


14.67 


10.11 


4.56** 


4. Ethical appeal 


12.53 


10.22 


2.31* 


14.47 


10.56 


3.91** 


5. Emotional appeal 


12.73 


10.94 


1.79* 


14.13 


11.44 


2.69** 


6. Treatment of 


11.07 


9.06 


2.01 


13.20 


10.29 


2.91* 


Opposing 

Views 


7. Organization 


12.67 


10.89 


1.78* 


14.00 


9.44 


4.56** 


8. Syntax 


13.60 


11.06 


2.54* 


14.33 


10.78 


3.55** 


9. Grammar 


13.20 


10.89 


2.31* 


14.27 


10.78 


3.49** 



*p<.05. **£<.001 

In addition, students in the online section showed a significantly greater amount of change in logical 
appeal, ethical appeal, and organization than did the on-campus students. From the data, it would appear that 
students in the online section developed better audience awareness skills, such as use of logical, ethical, and 
emotional appeals, and treatment of opposing views, as the semester progressed, while the corresponding on-campus 
students did not. Neither section improved significantly in organization, syntax, or grammar. Scores on these three 
elements actually decreased for the on-campus section, although the decrease is not significant. 

Age as a Confounding Independent Variable in Audience Awareness Rubric Results 

Age had a moderate relationship to audience awareness scores on the first paper (R= 0.452, p < .01) and on 
the final paper (R = 0.522, p < .01), and a moderate relationship to the final course grade (R = 0.437', p < .05). When 
age was used as a covariate, we still found significant differences between the two sections with respect to ethical 
appeal, logical appeal, and organization. 

Satisfaction Level Results 

Students in both sections reported positive experiences with their respective classes, with similarly 
favorable evaluations of the instructor, with the on-campus section evaluations being slightly more favorable, but 
not statistically significant. Rating for the instructor in the on-campus section was an A, and in the online section, an 
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A-. One area in which students in the two sections differed significantly was in their perceptions of the course 
workload. Students in the online section perceived the workload as being more difficult (M ” 6.23) than did the on- 
campus students (M = 5.14), even though the syllabus, schedule, course assignments, and deadlines were identical 
for the two classes, and the actual workload was identical. One explanation is that online students had significantly 
lower expectations of time to be spent studying as well. It could be that the contrast between their perception of the 
workload and the actual workload made the actual workload seem heavier. But in spite of the fact that the online 
students believed that they had a higher workload than students did in the on-campus class, online students 
overwhelmingly indicated that they would prefer to take an online class to an on-campus writing class. 

Limitations 

The most important limitation to this study was the fact that students self-selected into the networked and 
online classes, and thus did not represent a truly random sampling of populations. Additionally, the small sample 
size makes generalization speculative. However, as a preliminary study of online versus on-campus writing classes, 
the research reported in this paper was fruitful for us and provides a sound basis for further exploration on 
collaboration and audience awareness in computer-mediated freshman writing classes. 

Conclusions 

The results of this study indicate that students who enroll in an online class have characteristics that 
differentiate them from students who don’t select into an online class. Another conclusion that we can draw from 
this study is that online students tend to be more independent learners, valuing collaboration less than do on-campus 
students. Additionally, as indicated by the higher scores in audience awareness on students’ final papers in the 
online section, even when the scores were covaried for age, students can learn as well in an online class as in an on- 
campus class. Online interaction among students thus appears to increase audience awareness in students’ 
argumentative papers. This in itself is a surprising finding. We speculated that students in the on-campus section 
would have the advantage of having online interaction along with face-to-face interaction. Nonetheless, this 
advantage did not translate significantly into improvement in students’ papers while improvement among the online 
students was significant in several elements of audience awareness. With the increasing emphasis on distance 
learning, this study points to a need for further investigation of the pedagogical implications of teaching 
undergraduate writing courses online. 
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